Generalized Weiszfeld algorithms for Lq optimization

In many computer vision applications, a desired model of some type is computed by minimizing a cost function based on several measurements. Typically, one may compute the model that minimizes the L₂ cost, that is the sum of squares of measurement errors with respect to the model. However, the Lq solution which minimizes the sum of the qth power of errors usually gives more robust results in the presence of outliers for some values of q, for example, q = 1. The Weiszfeld algorithm is a classic algorithm for finding the geometric L1 mean of a set of points in Euclidean space. It is provably optimal and requires neither differentiation, nor line search. The Weiszfeld algorithm has also been generalized to find the L1 mean of a set of points on a Riemannian manifold of non-negative curvature. This paper shows that the Weiszfeld approach may be extended to a wide variety of problems to find an Lq mean for 1 ≤ q <; 2, while maintaining simplicity and provable convergence. We apply this problem to both single-rotation averaging (under which the algorithm provably finds the global Lq optimum) and multiple rotation averaging (for which no such proof exists). Experimental results of Lq optimization for rotations show the improved reliability and robustness compared to L₂ optimization.This research has been funded by National ICT Australia

L1-rotation averaging using the Weiszfeld algorithm

We consider the problem of rotation averaging under the L1 norm. This problem is related to the classic Fermat-Weber problem for finding the geometric median of a set of points in IRn. We apply the classical Weiszfeld algorithm to this problem, adapting it iteratively in tangent spaces of SO(3) to obtain a provably convergent algorithm for finding the L1 mean. This results in an extremely simple and rapid averaging algorithm, without the need for line search. The choice of L1 mean (also called geometric median) is motivated by its greater robustness compared with rotation averaging under the L2 norm (the usual averaging process). We apply this problem to both single-rotation averaging (under which the algorithm provably finds the global L1 optimum) and multiple rotation averaging (for which no such proof exists). The algorithm is demonstrated to give markedly improved results, compared with L2 averaging. We achieve a median rotation error of 0.82 degrees on the 595 images of the Notre Dame image set

LQ-bundle adjustment

In this paper we propose a method to solve for an Lq solution of bundle adjustment, a non-linear parameter estimation problem. Given a set of images of a scene, bundle adjustment simultaneously estimates camera parameters and 3D structure of the scene. Generally, a least squares criterion is minimized by using the Levenberg-Marquardt (LM) method, a non-linear least squares optimization method. It is known that the least squares methods are not robust to outliers, even a single outlier can deviate the solution from its true value. Therefore, we propose a method to minimize an Lq cost function, for 1 ≤ q < 2. The Lq cost function minimizes the sum of the q-th power of errors. The proposed method has an advantage of using the Levenberg-Marquardt (LM) method to find a robust solution of the problem. Our experimental results confirm that the proposed method is more robust to outliers than the standard least squares method

Generalized weiszfeld algorithms for lq optimization

In many computer vision applications, a desired model of some type is computed by minimizing a cost function based on several measurements. Typically, one may compute the model that minimizes the L2 cost, that is the sum of squares of measurement errors with respect to the model. The simplicity of L2 methods is well appreciated, but they suffer from a drawback when it comes to robustness against outliers; even a single outlier may change the L2 solution drastically. This thesis is primarily focused on the development of simple and robust Lq optimization methods, for q ranging from 1 to 2 (excluding 2), for problems in the area of computer vision. The proposed Lq optimization methods minimize the sum of the q-th power of errors and give more robust results than least squares methods in the presence of outliers. We particularly consider two classes of problems: Firstly, Lq-closest-point problems, where we seek a point for which the sum of the q-th power of distances from a given set of measurements is the minimum. Secondly, Lq non-linear parameter estimation problems, where parameters of a model are estimated by minimizing the sum of the q-th power of distances to a given set of points. The proposed Lq-closest-point algorithms are inspired by the Weiszfeld algorithm, a classic algorithm for finding the L1 mean of a set of points in a Euclidean space. The proposed Lq-closest-point algorithms inherit all the properties of the Weiszfeld algorithm, such as provable convergence to the global Lq minimum, analytical updates, simple to understand and easy to code. We specifically propose the following algorithms: First of all, we propose a generalization of the Weiszfeld algorithm to find the Lq mean of a set of points in a Euclidean space. We refer to this as the Lq Weiszfeld algorithm. We then propose a generalization of the Lq Weiszfeld algorithm to find the Lq mean of a set of points on a Riemannian manifold of non-negative sectional curvature and apply it to rotation averaging and Symmetric Positive-Definite matrices averaging problems. In addition to the proof of convergence, we relax the bounds on maximum distance between points on manifold to ensure convergence. Furthermore, we propose a generalization of the Lq Weiszfeld algorithm to find the Lq-closest-point to a set of affine subspaces, possibly of different dimensions, in a Euclidean space and apply it to the triangulation problem. In addition to the closest-point problems, we propose an algorithm to find an Lq solution of a non-linear parameter estimation problem, specifically the bundle adjustment problem. An advantage of the proposed algorithm is that an efficient least squares optimization method, namely the Levenberg-Marquardt method, is used to find a robust solution to the problem, even an Lq solution. In all the cases, our experimental results confirm the fact that in the presence of outliers the proposed Lq algorithms give superior results than least squares algorithms

Lq averaging for symmetric positive-definite matrices

We propose a method to find the Lq mean of a set of symmetric positive-definite (SPD) matrices, for 1≤q ≤2. Given a set of points, the Lq mean is defined as a point for which the sum of q-th power of distances to all the given points is minimum. The

Convergence of iteratively re-weighted least squares to robust M-estimators**

This paper presents a way of using the Iteratively Reweighted Least Squares (IRLS) method to minimize several robust cost functions such as the Huber function, the Cauchy function and others. It is known that IRLS (otherwise known as Weiszfeld) techniques are generally more robust to outliers than the corresponding least squares methods, but the full range of robust M-estimators that are amenable to IRLS has not been investigated. In this paper we address this question and show that IRLS methods can be used to minimize most common robust M-estimators. An exact condition is given and proved for decrease of the cost, from which convergence follows. In addition to the advantage of increased robustness, the proposed algorithm is far simpler than the standard L1 Weiszfeld algorithm. We show the applicability of the proposed algorithm to the rotation averaging, triangulation and point cloud alignment problems

L q-closest-point to affine subspaces using the generalized Weiszfeld Algorithm

This paper presents a method for finding an Lq - closest-point to a set of affine subspaces, that is a point for which the sum of the q-th power of orthogonal distances to all the subspaces is minimized, where 1 ≤ q < 2. We give a theoretical proof for the convergence of the proposed algorithm to a unique Lq minimum. The proposed method is motivated by the Lq Weiszfeld algorithm, an extremely simple and rapid averaging algorithm, that finds the Lq mean of a set of given points in a Euclidean space. The proposed algorithm is applied to the triangulation problem in computer vision by finding the Lq -closest-point to a set of lines in 3D. Our experimental results for the triangulation problem confirm that the Lq -closest-point method, for 1 ≤ q < 2, is more robust to outliers than the L2-closest-point methodThis research has been funded by National ICT Australia. National ICT Australia is funded by the Australian Government as represented by the Department of Broadband, Communications and the Digital Economy and the Australian Research Council through the ICT Centre of Excellence program